Live video broadcast method, live broadcast device and storage medium

ABSTRACT

The present disclosure provides a live video broadcast method performed by a terminal device. The method includes: receiving a live broadcast command to live broadcast a video game in real time, and creating a video buffer for the video game based on the live broadcast command; while playing the video game in real time, extracting video picture frames from the video game and storing them in the video buffer; collecting voice data by using a microphone of the terminal device, and synchronously synthesize the voice data and the video picture frames stored in the video buffer into a video streaming media file; and uploading the video streaming media file to a live broadcast server for live broadcasting the video streaming media file on other mobile devices.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/534,255, entitled “LIVE VIDEO BROADCAST METHOD, LIVE BROADCAST DEVICEAND STORAGE MEDIUM” filed Nov. 23, 2021, which is a continuation of U.S.application Ser. No. 16/351,267, entitled “LIVE VIDEO BROADCAST METHOD,LIVE BROADCAST DEVICE AND STORAGE MEDIUM” filed Mar. 12, 2019, which isa continuation of PCT Application PCT/CN2017/111535, entitled “LIVEVIDEO BROADCAST METHOD, LIVE BROADCAST DEVICE AND STORAGE MEDIUM” filedNov. 17, 2017, which claims priority to Chinese Patent Application No.201611073970.X, filed with the Chinese Patent Office on Nov. 29, 2016,and entitled “LIVE VIDEO BROADCAST METHOD AND APPARATUS”, all of whichare incorporated by reference in their entirety.

FIELD OF THE TECHNOLOGY

Embodiments of the present disclosure relate to the Internet field, andin particular, to a live video broadcast method and apparatus, and astorage medium.

BACKGROUND OF THE DISCLOSURE

With the development of science and technology, people have increasingmethods to play games, for example, a computer game or a mobile game.Meanwhile, a game user may share a computer game picture or a mobilegame picture, that is, performs live broadcasting of and shares thecomputer game or the mobile game.

SUMMARY

Embodiments of the present disclosure provide a live video broadcastmethod and apparatus, and a storage medium.

A first aspect of the present disclosure provides a live video broadcastmethod performed at a first mobile device having one or more processorsand memory storing a plurality of programs to be executed by the one ormore processors, the method comprising:

-   -   receiving, by the first mobile device, a live broadcast command        to live broadcast a video game being played by a user on the        first mobile device, wherein the video game generates, in real        time, game pictures using a game engine running on the first        mobile device;    -   creating a video buffer based on the live broadcast command for        storing one or more of the game pictures of the video game in        real time;    -   binding the video buffer with the game engine;    -   while the user is playing the video game:        -   detecting and extracting a plurality of video picture frames            from the video game using the game engine;        -   storing the plurality of video picture frames in the video            buffer;    -   transferring the plurality of video picture frames from the        video buffer into a frame buffer;    -   displaying the plurality of video picture frames in the frame        buffer on the display of the first mobile device;        -   collecting voice data using the microphone;        -   synchronously synthesizing the voice data and the plurality            of video picture frames stored in the video buffer into a            video streaming media file corresponding to the video game;            and    -   uploading the video streaming media file to a live broadcast        server, so that the live broadcast server performs live        broadcasting of the video streaming media file in response to        one or more live streaming requests from other mobile devices.

A second aspect of the present disclosure provides a first mobile devicehaving one or more processors, memory coupled to the one or moreprocessors, and a plurality of computer-readable instructions that, whenexecuted by the one or more processors, cause the first mobile device toperform the aforementioned live video broadcast method.

A third aspect of the present disclosure provides a non-transitorycomputer readable storage medium storing a plurality of instructions inconnection with a server having one or more processors. The plurality ofinstructions, when executed by the one or more processors, cause theserver to perform the aforementioned live video broadcast method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a live video broadcast method according to anembodiment of the present disclosure;

FIG. 2 is a flowchart of a live video broadcast method according to anembodiment of the present disclosure;

FIG. 3 is a flowchart of step S201 of a live video broadcast methodaccording to an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of a live video broadcastapparatus according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of a live video broadcastapparatus according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a buffer creation module ofa live video broadcast apparatus according to an embodiment of thepresent disclosure;

FIG. 7 is a schematic structural diagram of a loading method replacementunit of a buffer creation module of a live video broadcast apparatusaccording to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a live broadcast servercorresponding to a live video broadcast apparatus according to anembodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of a live broadcast servercorresponding to a live video broadcast apparatus according to anembodiment of the present disclosure;

FIG. 10 is a sequence diagram of a working process of a specificembodiment of a live video broadcast method and apparatus according toan embodiment of the present disclosure; and

FIG. 11 is a schematic structural diagram of an operating environment ofan electronic device in which a live video broadcast apparatus islocated according to an embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

Referring to the drawings, same components are represented by samecomponent symbols. The principle of the present disclosure isillustrated by an application in a suitable computing environment. Thefollowing description is based on the illustrated specific embodiment ofthe present disclosure, which should not be construed as limiting otherspecific embodiments not described in detail herein.

In the description below, the specific embodiments of the presentdisclosure will be described with reference to steps and signs ofoperations that are performed by one or more computers, unless indicatedotherwise. Therefore, it will be understood that such steps andoperations, which are at times referred to as being computer-executed,include the manipulation by the processing unit of the computer ofelectrical signals representing data in a structured form. Thismanipulation transforms the data or maintains it at locations in thememory system of the computer, which reconfigures or otherwise altersthe operation of the computer in a manner well understood by a personskilled in the art. The data structures where data is maintained arephysical locations of the memory that have particular properties definedby the format of the data. However, while the principle of the presentdisclosure is being described in the foregoing text, it is not meant tobe limiting as a person skilled in the art will appreciate that thevarious steps and operations described hereinafter may also beimplemented in hardware.

The present disclosure provides a live video broadcast method andapparatus. The live video broadcast method and apparatus in the presentdisclosure may be applied to various mobile terminals, so that themobile terminal performs live video broadcast on the mobile terminal.The mobile terminal may be a terminal device such as a mobile terminalhaving an IOS Apple system. A mobile terminal in the present disclosuremay perform live video broadcast on the mobile terminal anytime at anyplace, and present barrage comments (e.g., also referred to as comments)or a comment for a video on another mobile terminal in a timely manner.In some embodiments, barrage comments of a video (e.g., a live video)refer to live comments that are posted by one or more or all viewers ofthe video in real time as the viewers watch the video on theirrespective terminal devices. In some embodiments, barrage commentsinclude a plurality of comments from multiple viewers that are displayedin real time while the live video is displayed on a first client device.For example, the plurality of comments are displayed to overlay thevideo. In some embodiments, the barrage comments are displayed with anytype of suitable animation, such as scrolling left or right, sliding upand down, flying up and down, etc. In some embodiments, the barragecomments are displayed simultaneously as the corresponding video contentare being displayed on the first client device. For example, commentsabout a certain scene of a live video posted by multiple viewers fromtheir respective terminal devices are displayed on the first clientdevice while image frames associated with the certain scene are beingbroadcasted on the first client device. The terminal device includes amemory and a processor. The memory stores an instruction that can beexecuted by the processor. The processor implements the live videobroadcast method in the following embodiments by executing theinstruction.

Referring to FIG. 1 , FIG. 1 is a flowchart of an embodiment of a livevideo broadcast method according to the present disclosure. The livevideo broadcast method in this embodiment may be implemented by usingthe foregoing mobile terminal. The live video broadcast method in thisembodiment includes:

Step S101. Receive a live broadcast command, and create a video bufferbased on the live broadcast command.

Step S102. Bind the video buffer with a picture drawing environment, thepicture drawing environment being configured to detect and extract avideo picture frame.

Step S103. Detect and extract the video picture frame by using thepicture drawing environment (that is, detect and extract a video pictureby using a picture drawing module), and perform a storage operation onall video picture frames by using the video buffer.

Step S104. Collect an external voice by using a microphone, andsynchronously synthesize the external voice and the video picture framesinto a video streaming media file.

Step S105. Upload the video streaming media file to a live broadcastserver, so that the live broadcast server performs live broadcasting.

A specific procedure of each step of the live video broadcast method inthis embodiment is described in detail below.

In step S101, a live video broadcast apparatus (a mobile terminal)receives a live broadcast command of a user. The live broadcast commandis a command that the user requests to upload a current video picture ofthe live video broadcast apparatus, for example, a game picture of acurrently running game, to the live broadcast server, to perform livebroadcasting of the video picture.

Subsequently, the live video broadcast apparatus creates a video bufferbased on the live broadcast command. The video buffer is used to performa storage operation on a video picture frame of a current video.Subsequently, step S102 is performed.

In step S102, the live video broadcast apparatus binds the video buffercreated in step S101 and the picture drawing environment. The picturedrawing environment is configured to detect and extract the videopicture frame of the current video. The picture drawing environment maybe a game drawing environment such as an OpenGL (Open Graphics Library)context. After binding the video buffer with the picture drawingenvironment, the live video broadcast apparatus may extract the videopicture frame detected by the picture drawing environment into the videobuffer. Subsequently, step S103 is performed.

It may be understood that the picture drawing environment a softwaresystem having a picture drawing function, an applet, or the like.Therefore, the picture drawing environment may alternatively be referredto as “the picture drawing module”. Therefore, step S102 mayalternatively be described as “Bind the video buffer with the picturedrawing module”.

In step S103, the live video broadcast apparatus detects and extractsthe video picture frame by using the picture drawing environment, andsubsequently, the live video broadcast apparatus performs the storageoperation on all the video picture frames by using the video buffercreated in step S101. Subsequently, step S104 is performed.

It may be understood that, that the video buffer is used to perform thestorage operation on all the video picture frames is actually that allthe video picture frames extracted by the picture drawing environmentare stored in the video buffer.

In step S104, the live video broadcast apparatus collects an externalvoice of the live video broadcast apparatus by using a microphone of thelive video broadcast apparatus. The external voice may include a soundgiven by a horn of the live video broadcast apparatus, a voice of avideo user, and the like. Such external voice may both include a videosound, and may also include an explanation of a video by a video user,or even surrounding environment music of the video user. Certainly, thevideo user herein may alternatively disable the video sound as required,and only reserve the voice of the video user and the like.

Subsequently, the live video broadcast apparatus synchronouslysynthesizes the collected external voice and the video picture framecollected in step S103, to generate the video streaming media file, suchas an FLV (Flash Video) streaming media file. The video user may controlthe size of the video streaming media file by controlling a frame rateof the generated video streaming media file. In this way, video picturelive broadcast content including both video information and audioinformation is obtained. Subsequently, step S105 is performed.

In step S105, the live video broadcast apparatus uploads the videostreaming media file synthesized in step S104 to the live broadcastserver, so that the live broadcast server may use the video streamingmedia file to perform live broadcasting of the video.

In this way, a live video broadcast process of the live video broadcastmethod in this embodiment is completed.

According to the live video broadcast method in this embodiment, thevideo picture frame and the external voice are synthesized byconfiguring an additional video buffer, so that the live broadcastserver processed a synthesized video streaming media file. The videouser may perform live broadcasting of a mobile video anytime.

Referring to FIG. 2 , FIG. 2 is a flowchart of an embodiment of a livevideo broadcast method according to the present disclosure. The livevideo broadcast method in this embodiment may be implemented by usingthe foregoing mobile terminal. The live video broadcast method in thisembodiment includes:

Step S201. Receive a live broadcast command, and create a video bufferbased on the live broadcast command.

Step S202. Bind the video buffer with a picture drawing environment, thepicture drawing environment being configured to detect and extract avideo picture frame.

Step S203. Detect and extract the video picture frame by using thepicture drawing environment, and perform a storage operation on allvideo picture frames by using the video buffer.

Step S204. Collect an external voice by using a microphone, andsynchronously synthesize the external voice and the video picture framesinto a video streaming media file.

Step S205. Upload the video streaming media file to a live broadcastserver, so that the live broadcast server performs live broadcasting.

Step S206. Update the video picture frame to a frame buffer.

Step S207. Perform broadcasting of the video picture frame in the framebuffer by using a screen of the live video broadcast apparatus.

A specific procedure of each step of the live video broadcast method inthis embodiment is described in detail below.

In step S201, the live video broadcast apparatus receives the livebroadcast command, and creates the video buffer based on the livebroadcast command. Specifically, referring to FIG. 3 , FIG. 3 is aflowchart of step S201 in this embodiment of the live video broadcastmethod according to the present disclosure. Step S201 may include thefollowing steps.

Step S301: The live video broadcast apparatus invokes a pre-configuredmounting component based on the live broadcast command. On existing livevideo broadcast apparatuses, especially on a live video broadcastapparatus of an IOS Apple system, a bottom layer of a game engine usesboth OpenGL and Metal. Therefore, the preconfigured mounting componentmay be used to detect game picture loading of the game engine.Subsequently, step S302 is performed.

Step S302. When the live video broadcast apparatus detects game pictureloading of a game engine, replace a picture loading method by using themounting component invoked in step S301.

Specifically, the live video broadcast apparatus detects whether thepicture loading method is invoked, and if the picture loading method isinvoked, replaces a picture drawing method in the picture loadingmethod. For example, an occasion of invoking a static method may betriggered by loading Class of a game to a memory, to replace a picturedrawing method, for example, presentRenderbuffer, in the picture loadingmethod, so that an OpenGL (Open Graphics Library) context of a gamedrawing environment may detect updating of a video picture frame. Inthis way, the live video broadcast apparatus may detect each videopicture frame of a current video by using a replaced picture loadingmethod. Subsequently, step S303 is performed.

Step S303. The live video broadcast apparatus creates a correspondingvideo buffer based on the picture loading method replaced in step S302,to perform a storage operation on a video picture frame extracted by apicture drawing environment. Subsequently, step S202 is performed.

In step S202, the live video broadcast apparatus binds the video buffercreated in step S201 with the picture drawing environment. After bindingthe video buffer with the picture drawing environment, the live videobroadcast apparatus can extract the video picture frame detected by thepicture drawing environment into the video buffer. Subsequently, stepS203 is performed.

In step S203, the live video broadcast apparatus detects and extractsthe video picture frame by using the picture drawing environment, andsubsequently, the live video broadcast apparatus performs the storageoperation on all the video picture frames by using the video buffercreated in step S201. Subsequently, step S204 and step S206 areperformed.

In step S204, the live video broadcast apparatus collects the externalvoice of the live video broadcast apparatus by using the microphone ofthe live video broadcast apparatus. Subsequently, the live videobroadcast apparatus synchronously synthesizes the collected externalvoice and the video picture frame collected in step S203, to generatethe video streaming media file, such as an FLV (Flash Video) streamingmedia file. A user may control the size of the video streaming mediafile by controlling a frame rate of the generated video streaming mediafile. In this way, video picture live broadcast content including bothvideo information and audio information is obtained. Subsequently, stepS202 is performed.

In step S205, the live video broadcast apparatus uploads the videostreaming media file synthesized in step S204 to the live broadcastserver, so that the live broadcast server may use the video streamingmedia file to perform live broadcasting of the video.

Specifically, when performing live broadcasting of the video streamingmedia file, the live broadcast server may receive comment information ofanother terminal; and subsequently synthesize the comment informationwith the video streaming media file, to obtain a live streaming mediafile, and finally the live broadcast server performs a playbackoperation on the live streaming media file. In this way, comments ofothers users may be presented in a timely manner during livebroadcasting of the mobile video.

In addition, the live broadcast server may further receive barragecomments information from another terminal. Subsequently, whenperforming the playback operation on the video streaming media file, thelive broadcast server synchronously superimpose the comments informationon a playback screen. In this way, barrage comments information ofothers users may be presented in a timely manner during livebroadcasting of the mobile video.

In step S206, in addition to uploading the video picture to the livebroadcast server, presentation of the video picture on the live videobroadcast apparatus also needs to be ensured continuously. Therefore,the live video broadcast apparatus further needs to update the videopicture frame extracted in step S203 from the video buffer to the framebuffer. The frame buffer is configured to perform the storage operationon the video picture frame presented on the screen of the live videobroadcast apparatus. Subsequently, step S207 is performed.

It may be understood that, in fact, S206 is that the video picture framein the video buffer is extracted and stored in the frame buffer, so thatthe video picture frame in the frame buffer may be drawn onto a displayinterface. That is, the video picture frame in the frame buffer isdisplayed on a terminal screen, to implement playback of the videopicture.

In step S207, the live video broadcast apparatus performs playback ofthe video picture frame in the frame buffer by using the screen. In thisway, normal display of the video picture on the live video broadcastapparatus is also ensured.

In this way, a live video broadcast process of the live video broadcastmethod in this embodiment is completed.

Based on the previous embodiment, in the live video broadcast method inthis embodiment, the mounting component is disposed, to further improvestability of obtaining the video picture frame; setting of the videobuffer and the frame buffer ensures normal display of a live broadcastpicture and the video picture; and the live broadcast server may furtherpresent barrage comments and a comment of another user in a timelymanner.

The present disclosure further provides a live video broadcastapparatus. Referring to FIG. 4 , FIG. 4 is a schematic structuraldiagram of a live video broadcast apparatus according to an embodimentof the present disclosure. The live video broadcast apparatus in thisembodiment may use the live video broadcast method in the foregoingfirst embodiment. The live video broadcast apparatus 40 in thisembodiment includes: one or more memories; and one or more processors,the one or more memories storing one or more command modules, configuredto be executed by the one or more processors, the one or more commandmodules including a buffer creation module 41, a buffer binding module42, a picture frame storage module 43, a synthesizing module 44, and anuploading module 45.

The buffer creation module 41 is configured to: receive a live broadcastcommand, and create a video buffer based on the live broadcast command.The buffer binding module 42 is configured to bind the video buffer witha picture drawing environment, where the picture drawing environment isconfigured to detect and extract a video picture frame. The pictureframe storage module 43 is configured to: detect and extract the videopicture frame by using the picture drawing environment, and perform astorage operation on all video picture frames by using the video buffer.The synthesizing module 44 is configured to: collect an external voiceby using a microphone, and synchronously synthesize the external voiceand the video picture frames into a video streaming media file. Theuploading module 45 is configured to upload the video streaming mediafile to a live broadcast server, so that the live broadcast serverperforms live broadcasting.

When the live video broadcast apparatus 40 is used, the buffer creationmodule 41 receives a live broadcast command of a user. The livebroadcast command is a command that the user requests to upload acurrent video picture of the live video broadcast apparatus, forexample, a game picture of a currently running game, to the livebroadcast server, to perform live broadcasting of the video picture.

Subsequently, the buffer creation module 41 creates a video buffer basedon the live broadcast command. The video buffer is used to perform astorage operation on a video picture frame of a current video.

Subsequently, the buffer binding module 42 binds the video buffercreated by the buffer creation module 41 with a picture drawingenvironment. The picture drawing environment is configured to detect andextract the video picture frame of the current video. The picturedrawing environment may be a game drawing environment such as an OpenGL(Open Graphics Library) context. After binding the video buffer with thepicture drawing environment, the live video broadcast apparatus mayextract the video picture frame detected by the picture drawingenvironment into the video buffer.

Subsequently, the picture frame storage module 43 detects and extractsthe video picture frame by using the picture drawing environment.Subsequently, the picture frame storage module 43 performs a storageoperation on all video picture frames by using the video buffer createdby the buffer creation module.

Subsequently, the synthesizing module 44 collects an external voice ofthe live video broadcast apparatus by using the microphone of the livevideo broadcast apparatus 40. The external voice may include a soundgiven by a horn of the live video broadcast apparatus, a voice of avideo user, and the like. Such external voice may both include a videosound, and may also include an explanation of a video by a video user,or even surrounding environment music of the video user. Certainly, thevideo user herein may alternatively disable the video sound as required,and only reserve the voice of the video user and the like.

Subsequently, the synthesizing module 44 synchronously synthesizes thecollected external voice and the video picture frame collected by thepicture frame storage module, to generate the video streaming mediafile, such as an FLV (Flash Video) streaming media file. The video usermay control the size of the video streaming media file by controlling aframe rate of the generated video streaming media file. In this way,video picture live broadcast content including both video informationand audio information is obtained.

Finally, the uploading module 45 uploads the video streaming media filesynthesized by the synthesizing module 44 to the live broadcast server,so that the live broadcast server may perform live broadcasting of thevideo by using the video streaming media file.

In this way, a live video broadcast process of the live video broadcastapparatus 40 in this embodiment is completed.

According to the live video broadcast apparatus in this embodiment, thevideo picture frame and the external voice are synthesized byconfiguring an additional video buffer, so that the live broadcastserver processed a synthesized video streaming media file. The videouser may perform live broadcasting of a mobile video anytime.

Referring to FIG. 5 , FIG. 5 is a schematic structural diagram of a livevideo broadcast apparatus according to an embodiment of the presentdisclosure. The live video broadcast apparatus in this embodiment mayuse the live video broadcast method in the foregoing second embodiment.The live video broadcast apparatus 50 in this embodiment includes:

-   -   one or more memories; and    -   one or more processors, the one or more memories storing one or        more command modules, configured to be executed by the one or        more processors,    -   the one or more command modules including a buffer creation        module 51, a buffer binding module 52, a picture frame storage        module 53, a synthesizing module 54, an uploading module 55, a        frame buffer updating module 56, and a terminal playback module        57.

The buffer creation module 51 is configured to: receive a live broadcastcommand, and create a video buffer based on the live broadcast command.The buffer binding module 52 is configured to bind the video buffer witha picture drawing environment, where the picture drawing environment isconfigured to detect and extract a video picture frame. The pictureframe storage module 53 is configured to: detect and extract the videopicture frame by using the picture drawing environment, and perform astorage operation on all video picture frames by using the video buffer.The synthesizing module 54 is configured to: collect an external voiceby using a microphone, and synchronously synthesize the external voiceand the video picture frames into a video streaming media file. Theuploading module 55 is configured to upload the video streaming mediafile to a live broadcast server, so that the live broadcast serverperforms live broadcasting. The frame buffer updating module 56 isconfigured to update the video picture frame to a frame buffer. Theterminal playback module 57 is configured to perform playback of thevideo picture frame in the frame buffer by using a screen of a mobileterminal.

Referring to FIG. 6 , FIG. 6 is a schematic structural diagram of abuffer creation module of a live video broadcast apparatus according toan embodiment of the present disclosure. The buffer creation module 51includes a mounting component invoking unit 61, a loading methodreplacement unit 62, and a buffer creation unit 63.

The mounting component invoking unit 61 is configured to invoke apre-configured mounting component based on a live broadcast command. Theloading method replacement unit 62 is configured to replace a pictureloading method by using the mounting component. The buffer creation unit63 creates a corresponding video buffer based on a replaced pictureloading method.

Referring to FIG. 7 , FIG. 7 is a schematic structural diagram of aloading method replacement unit of a buffer creation module of a livevideo broadcast apparatus according to an embodiment of the presentdisclosure. The loading method replacement unit 62 includes a detectionsubunit 71 and a loading method replacement subunit 72.

The detection subunit 71 is configured to detect whether a pictureloading method is invoked. The loading method replacement subunit 72 isconfigured to: if the picture loading method is invoked, replace apicture drawing method in the picture loading method, so that updatingof a video picture frame is detected by using a picture drawingenvironment.

When the live video broadcast apparatus 50 in this embodiment is used,the buffer creation module 51 first receives the live broadcast command,and creates the video buffer based on the live broadcast command.

Specifically, the mounting component invoking unit 61 of the buffercreation module 51 invokes the mounting component based on the livebroadcast command. On existing live video broadcast apparatuses,especially on a live video broadcast apparatus of an IOS Apple system, abottom layer of a game engine uses both OpenGL and Metal. Therefore, thepre-configured mounting component may be used to detect game pictureloading of the game engine.

When the live video broadcast apparatus 50 detects game picture loadingof the game engine, the loading method replacement unit 62 of the buffercreation module 51 immediately uses the mounting component invoked bythe mounting component invoking unit to replace the picture loadingmethod.

Specifically, the detection subunit 71 of the loading method replacementunit 62 detects whether the picture loading method is invoked. If thepicture loading method is invoked, the loading method replacementsubunit 72 of the loading method replacement unit 62 replaces thepicture drawing method in the picture loading method. For example, anoccasion of invoking a static method may be triggered by loading Classof a game to a memory, to replace the picture drawing method, forexample, presentRenderbuffer, in the picture loading method, so that anOpenGL (Open Graphics Library) context of a game drawing environment maydetect updating of a video picture frame. In this way, the live videobroadcast apparatus 50 may detect each video picture frame of a currentvideo by using the replaced picture loading method.

The buffer creation unit 63 of the buffer creation module 51 creates thecorresponding video buffer based on the picture loading method replacedby the loading method replacement unit 62, so as to perform the storageoperation on the video picture frame extracted by the picture drawingenvironment.

Subsequently, the buffer binding module 52 binds the video buffercreated by the buffer creation module 51 with the picture drawingenvironment. After binding the video buffer with the picture drawingenvironment, the live video broadcast apparatus 50 may extract the videopicture frame detected by the picture drawing environment into the videobuffer.

Subsequently, the picture frame storage module 53 detects and extractsthe video picture frame by using the picture drawing environment.Subsequently, the picture frame storage module 53 performs the storageoperation on all video picture frames by using the video buffer createdby the buffer creation module.

Subsequently, the synthesizing module 54 collects the external voice ofthe live video broadcast apparatus by using the microphone of the livevideo broadcast apparatus 50. Subsequently, the synthesizing module 54synchronously synthesizes the collected external voice and the videopicture frame collected by the picture frame storage module, to generatethe video streaming media file, such as an FLV (Flash Video) streamingmedia file. A user may control the size of the video streaming mediafile by controlling a frame rate of the generated video streaming mediafile. In this way, video picture live broadcast content including bothvideo information and audio information is obtained.

Finally, the uploading module 55 uploads the video streaming media filesynthesized by the synthesizing module 54 to the live broadcast server,so that the live broadcast server may perform live broadcasting of thevideo by using the video streaming media file.

Specifically, referring to FIG. 8 , FIG. 8 is a schematic structuraldiagram 1 of a live broadcast server corresponding to a live videobroadcast apparatus according to an embodiment of the presentdisclosure. The live broadcast server 80 includes a comment receivingmodule 81, a comment synthesizing module 82, and a playback module 83.

The comment receiving module 81 is configured to receive commentinformation of another terminal. The comment synthesizing module 82 isconfigured to synthesize the comment information and the video streamingmedia file, to obtain a live streaming media file. The playback module83 is configured to perform a playback operation on the live streamingmedia file.

When the live broadcast server 80 performs live broadcasting of thevideo streaming media file, the comment receiving module 81 may receivethe comment information of the another terminal. Subsequently, thecomment synthesizing module 82 synthesizes the comment information withthe video streaming media file, to obtain the live streaming media file.Finally, the playback module 83 performs the playback operation on thelive streaming media file. In this way, comments of others users may bepresented in a timely manner during live broadcasting of the mobilevideo.

In addition, further referring to FIG. 9 , FIG. 9 is a schematicstructural diagram 2 of a live broadcast server corresponding to a livevideo broadcast apparatus according to an embodiment of the presentdisclosure. The live broadcast server 90 includes a barrage commentsreceiving module 91 and a playback module 92. The barrage commentsreceiving module 91 is configured to receive barrage commentsinformation of another terminal. The playback module 92 is configured toperform a playback operation on the video streaming media file, andsynchronously superimpose the barrage comments information on a playbackscreen.

The barrage comments receiving module 91 receives the barrage commentsinformation of the another terminal. Subsequently, the playback module92 performs the playback operation on the video streaming media file,and synchronously superimposes the barrage comments information on theplayback screen. In this way, barrage comments information of othersusers may be presented in a timely manner during live broadcasting ofthe mobile video.

Meanwhile, in addition to uploading the video picture to the livebroadcast server, presentation of the video picture on the live videobroadcast apparatus also needs to be ensured continuously. Therefore,the frame buffer updating module 56 of the live video broadcastapparatus 50 further needs to update the video picture frame extractedby the picture frame storage module from the video buffer to the framebuffer. The frame buffer is configured to perform a storage operation onthe video picture frame presented on the screen of the live videobroadcast apparatus.

Finally, the terminal playback module 57 performs playback of the videopicture frame in the frame buffer by using the screen. In this way,normal display of the video picture on the live video broadcastapparatus is also ensured.

In this way, a live video broadcast process of the live video broadcastapparatus 50 in this embodiment is completed.

Based on the foregoing first embodiment, in the live video broadcastapparatus in this embodiment, the mounting component is disposed, tofurther improve stability of obtaining the video picture frame; settingof the video buffer and the frame buffer ensures normal display of alive broadcast picture and the video picture; and the live broadcastserver may further present barrage comments and a comment of anotheruser in a timely manner.

A specific working principle of a live video broadcast method andapparatus of the present disclosure is described below by using aspecific embodiment. Referring to FIG. 10 , FIG. 10 is a sequencediagram of a working process of the specific embodiment of the livevideo broadcast method and apparatus according to the presentdisclosure. The live video broadcast apparatus may be disposed in acorresponding mobile terminal having an IOS Apple system. The mobileterminal includes: a game drawing environment configured to detect andextract a game picture frame, a mounting component invoking the gamedrawing environment, a video buffer storing a game picture frame forlive broadcasting, a frame buffer storing a game picture frame displayedon the mobile terminal, a display screen that is on the mobile terminaland on which the game picture frame is displayed, a microphonecollecting an external voice, and a synchronous synthesizing threadsynthesizing the external voice and the game picture frame in the videobuffer.

A live video broadcast process of this specific embodiment includes:

Step S1001. After receiving a game live broadcast command of a user, themobile terminal invokes the corresponding mounting component.

Step S1002. The mounting component creates the video buffer by replacinga game picture loading method.

Step S1003. Bind the video buffer with the game drawing environment.

Step S1004. The game drawing environment detects the game picture frame,and stores the game picture frame in the video buffer.

Step S1005. The video buffer sends the game picture frame to thesynchronous synthesizing thread, and the microphone also sends thecollected external voice to the synchronous synthesizing thread at thesame time.

Step S1006. The synchronous synthesizing thread synchronouslysynthesizes the external voice and the game picture frame, to form anFLV streaming media file; and sends the FLV streaming media file to thelive broadcast server to perform live broadcasting of a game video.

Step S1007. The video buffer sends the game picture frame to the framebuffer.

Step S1008. The display screen displays the game picture frame in theframe buffer, thereby ensuring normal display of a game picture on themobile terminal.

In this way, according to the live video broadcast method and apparatusin this specific embodiment, live broadcast display of the game pictureon the live broadcast server and local display of the game picture onthe mobile terminal are completed.

According to the live video broadcast method and apparatus of thepresent disclosure, the video picture frame and the external voice aresynthesized by configuring an additional video buffer, so that the livebroadcast server processes a synthesized video streaming media file. Avideo user may perform live broadcasting of a mobile terminal any time,and the live broadcast server may present barrage comments and a commentof another user in a timely manner, to resolve technical problems thatlive broadcasting of the mobile video cannot be performed any timeaccording to an existing live video broadcast method and apparatus, andthe barrage comments and the comments of other users cannot be presentedin a timely manner during live broadcasting of the mobile video.

The terms, such as “component”, “module”, “system”, “interface”, and“process”, used in the present disclosure generally indicate acomputer-related entity: hardware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to, a process running on a processor, aprocessor, an object, an executable application, an executed thread, aprogram, and/or a computer. With reference to the drawings, anapplication running on a controller and the controller may both becomponents. One or more components may be in an executed process and/orthread and the components may be located on one computer and/ordistributed between or among two or more computers.

FIG. 11 and the following discussions provide short and summarydescriptions of a working environment of an electronic device in whichthe live video broadcast apparatus of the present disclosure is located.The working environment in FIG. 11 is only an instance of a suitableworking environment, and is not intended to suggest any limitation to ascope of a purpose or a function of the working environment. Theinstance of an electronic device 1112 includes, but is not limited to, awearable device, a head mounted device, a medical health platform, apersonal computer, a server computer, a handheld or laptop device, amobile device (for example, a mobile phone, a personal digital assistant(PDA), and a media player), a multiprocessor system, a consumption-basedelectronic device, a minicomputer, a mainframe computer, a distributedcomputing environment including the foregoing any system or device, andthe like.

Although not required, this embodiment is described under a generalbackground that “a computer-readable instruction” is executed by one ormore electronic devices. The computer-readable instruction may bedistributed by a computer-readable medium (discussed below). Thecomputer readable instruction may be implemented as a program module,for example, a function, an object, an application programming interface(API), or a data structure for performing a specific task orimplementing a specific abstract data type. Typically, functions of thecomputer readable instruction may be randomly combined or distributed invarious environments.

FIG. 11 shows the instance of the electronic device 1112 including oneor more embodiments of the live video broadcast apparatus of the presentdisclosure. In a configuration, the electronic device 1112 includes atleast one processing unit 1116 and a memory 1118. Based on an exactconfiguration and type of the electronic device, the memory 1118 may bea volatile memory (for example, a RAM), a non-volatile memory (forexample, a ROM or a flash memory), or a combination thereof. Theconfiguration is represented by using a dashed line 1114 in FIG. 11 .

In another embodiment, the electronic device 1112 may include anadditional feature and/or function. For example, the device 1112 mayfurther include an additional storage apparatus (for example, aremovable/or non-removable storage apparatus), and includes, but is notlimited to, a magnetic storage apparatus, an optical storage apparatus,and the like. The additional storage apparatus is represented by using astorage apparatus 1120 in FIG. 11 . In an embodiment, acomputer-readable instruction used to implement one or more embodimentsprovided in the present disclosure may be stored in the storageapparatus 1120. The storage apparatus 1120 may further be configured tostore other computer-readable instructions for implementing an operatingsystem, an application program, and the like. For example, thecomputer-readable instruction may be loaded to the memory 1118, andexecuted by the processing unit 1116.

The term “computer-readable media” used in the present disclosureincludes a computer storage medium. The computer storage media includesvolatile and non-volatile media, and removable and non-removable mediaimplemented by using any method or technology used for storinginformation such as a computer-readable instruction or other data. Thememory 1118 and the storage apparatus 1120 are instances of the computerstorage media. The computer storage media includes, but is not limitedto a RAM, a ROM, an EEPROM, a flash memory or another storagetechnology, a CD-ROM, a digital versatile disc (DVD) or another opticalstorage apparatus, a cassette, a magnetic tape, a magnetic disk storagedevice, or any other media configured to store desired information andaccessed by the electronic device 1112. Such a computer storage mediummay be a part of the electronic device 1112.

The electronic device 1112 may further include a communicationsconnection 1126 allowing communication between the electronic device1112 and another device. The communications connection 1126 may include,but is not limited to, a modem, a network interface card (NIC), anintegrated network interface, RF transmitter/receiver, infrared port, aUSB connection, or another interface configured to connect theelectronic device 1112 to another electronic device. The communicationsconnection 1126 may include a wired connection or a wireless connection.The communications connection 1126 may transmit and/or receive acommunications medium.

The term “computer-readable media” may include the communicationsmedium. Typically, the communications medium includes acomputer-readable instruction or other data in a “modulated datasignal”, for example, a carrier or another transmission mechanism, andincludes any information transmission medium. The term “modulated datasignal” may include such a signal: One or more features of the signalare set or changed by encoding information into the signal.

The electronic device 1112 may include an input device 1124, forexample, a keyboard, a mouse, a stylus, a voice input device, a touchinput device, an infrared camera, a video input device, and/or any otherinput device. The device 1112 may further include an output device 1122,for example, one or more displays, a speaker, a printer, and/or anyother output device. The input device 1124 and the output device 1122may be connected to the electronic device 1112 through a wiredconnection, a wireless connection, or any combination thereof. In anembodiment, an input device or an output device of another electronicdevice may be used as the input device 1124 or the output device 1122 ofthe electronic device 1112.

The components of the electronic device 1112 may be connected by usingvarious interconnects (for example, a bus). Such interconnect mayinclude a peripheral component interconnect (PCI) (for example, a PCIexpress), a universal serial bus (USB), a live line (IEEE 1394), anoptical bus structure, and the like. In another embodiment, thecomponents of the electronic device 1112 may be interconnected by usinga network. For example, the memory 1118 may include a plurality ofphysical memory units located at different physical positions andinterconnected by using the network.

A person skilled in the art may be aware that a storage deviceconfigured to store the computer-readable instruction may be distributedacross the network. For example, the electronic device 1130 that may beaccessed by using a network 1128 may store a computer-readableinstruction used to implement one or more embodiments of the presentdisclosure. The electronic device 1112 may access the electronic device1130 and download a part or all of the computer-readable instruction forexecution. Alternatively, the electronic device 1112 may download aplurality of computer-readable instructions as required, or someinstructions may be executed by the electronic device 1112 and someinstructions may be executed by the electronic device 1130.

The present disclosure provides various operations of embodiments. In anembodiment, the one or more operations may constitute one or morecomputer-readable instructions stored on a computer-readable medium, andthe computer-readable instructions enable a computing device to performthe operations when the computer-readable instructions are executed byan electronic device. Describing a sequence of some or all operationsshall not be interpreted as implying that the operations must besequentially related. A person skilled in the art will understand analternative sequence having the benefits of the present specification.Moreover, it should be understood that not all operations necessarilyexist in each embodiment provided in the present disclosure.

Furthermore, although the present disclosure is shown and described byusing one or more implementation manners, a person skilled in the artmay conceive of equivalent variations and modifications based on readingand understanding on the specification and the accompany drawings. Thepresent disclosure includes all such variations and modifications, whichis only limited by the scope of the appended claims. In particularregard to the various functions performed by the foregoing components(such as elements and resources), terms used to describe such componentsare intended to correspond to any component (unless indicated otherwise)performing specified functions of the components (for example, thecomponents are equivalent in functions), even though structures of thefunctions are not equivalent to the disclosed structures of functions inthe exemplary implementation manners in the present disclosure shown inthe specification. In addition, although specific features of thespecification are disclosed with respect to only one of severalimplementation manners, the features may be combined with one or moreother features of other implementation manners that are desirable forand advantageous to a given or specific application. Moreover, when theterms “include”, “include”, “contain” and any variants thereof are usedin a specific implementation or the claims, the terms are intended tocover in a manner similar to “include”.

Functional units according to the embodiments of the present disclosuremay be integrated in one processing module or exist as separate physicalunits, or two or more units are integrated in one module. The integratedmodule may be implemented in a form of hardware, or may be implementedin a form of a software functional module. If implemented in the form ofsoftware functional modules and sold or used as an independent product,the integrated modules may also be stored in a computer-readable storagemedium. The aforementioned storage medium may be a read-only memory, amagnetic disk or an optical disc. The foregoing apparatuses or systemscan execute methods in corresponding process embodiments.

An embodiment of the present disclosure further provides a non-volatilecomputer-readable storage medium, storing a computer program, and whenthe program is executed by the processor, steps of the foregoing methodbeing implemented.

In conclusion, although the present disclosure is disclosed above byusing the embodiments, the sequence numbers such as “first” and “second”in the embodiments are used merely for ease of description, but do notlimit a sequence of the embodiments of the present disclosure. Moreover,the foregoing embodiments are not used to limit the present disclosure.A person of ordinary skill in the art may make various modifications andrefinements without departing from the spirit of the present disclosure.Therefore, the protection scope of the present disclosure shall besubject to scope defined by the claims.

What is claimed is:
 1. A live video broadcast method performed at aterminal device, the method comprising: receiving a live broadcastcommand to live broadcast a video game in real time; creating a videobuffer based on the live broadcast command for storing one or more gamepictures of the video game in real time; while playing the video game inreal time: extracting a plurality of video picture frames from the videogame; storing the plurality of video picture frames in the video buffer;displaying the plurality of video picture frames on a display of theterminal device; collecting voice data using a microphone of theterminal device in real time; synchronously synthesizing the voice dataand the plurality of video picture frames stored in the video bufferinto a video streaming media file corresponding to the video game; anduploading the video streaming media file to a live broadcast server forlive broadcasting the video streaming media file on other mobiledevices.
 2. The method according to claim 1, wherein the operation ofcreating a video buffer based on the live broadcast command comprises:invoking a pre-configured mounting component based on the live broadcastcommand; replacing a picture loading method using the mountingcomponent; and creating a corresponding video buffer based on thereplaced picture loading method.
 3. The method according to claim 2,wherein the operation of replacing a picture loading method using themounting component comprise: detecting whether the picture loadingmethod is invoked; and replacing a picture drawing method in the pictureloading method when the picture loading method is invoked, so as todetect updating of the video picture frame by using the game engine. 4.The method according to claim 1, wherein the live broadcast server isconfigured to perform the live broadcasting by: receiving commentinformation associated with the video game from the other mobiledevices; synthesizing the comment information and the video streamingmedia file, to obtain a live streaming media file with comments; andperforming a broadcast operation on the live streaming media file withcomments to the other mobile devices, wherein a respective mobile deviceis configured to display the comments from other mobile devices that arecurrently live broadcasting the video game.
 5. The method according toclaim 1, wherein the live broadcast server is configured to perform thelive broadcasting by: receiving barrage comments information associatedwith the video game from the mobile devices; performing a broadcastoperation on the video streaming media file; and synchronouslysuperimposing the barrage comments information on the associated videopicture frames of the video streaming media file on a playback screen ofthe terminal device.
 6. A terminal device, comprising: one or moreprocessors; memory coupled to the one or more processors; and aplurality of programs stored in the memory that, when executed by theone or more processors, cause the terminal device to perform a livevideo broadcast method including: receiving a live broadcast command tolive broadcast a video game in real time; creating a video buffer basedon the live broadcast command for storing one or more game pictures ofthe video game in real time; while playing the video game in real time:detecting and extracting a plurality of video picture frames from thevideo game; storing the plurality of video picture frames in the videobuffer; displaying the plurality of video picture frames in the videobuffer on a display of the terminal device; collecting voice data usinga microphone of the terminal device in real time; synchronouslysynthesizing the voice data and the plurality of video picture framesstored in the video buffer into a video streaming media filecorresponding to the video game; and uploading the video streaming mediafile to a live broadcast server for live broadcasting the videostreaming media file on other mobile devices.
 7. The terminal deviceaccording to claim 6, wherein the operation of creating a video bufferbased on the live broadcast command comprises: invoking a pre-configuredmounting component based on the live broadcast command; replacing apicture loading method using the mounting component; and creating acorresponding video buffer based on the replaced picture loading method.8. The terminal device according to claim 7, wherein the operation ofreplacing a picture loading method using the mounting componentcomprise: detecting whether the picture loading method is invoked; andreplacing a picture drawing method in the picture loading method whenthe picture loading method is invoked, so as to detect updating of thevideo picture frame by using the game engine.
 9. The terminal deviceaccording to claim 6, wherein the live broadcast server is configured toperform the live broadcasting by: receiving comment informationassociated with the video game from the other mobile devices;synthesizing the comment information and the video streaming media file,to obtain a live streaming media file with comments; and performing abroadcast operation on the live streaming media file with comments tothe other mobile devices, wherein a respective mobile device isconfigured to display the comments from other mobile devices that arecurrently live broadcasting the video game.
 10. The terminal deviceaccording to claim 6, wherein the live broadcast server is configured toperform the live broadcasting by: receiving barrage comments informationassociated with the video game from the mobile devices; performing abroadcast operation on the video streaming media file; and synchronouslysuperimposing the barrage comments information on the associated videopicture frames of the video streaming media file on a playback screen ofthe terminal device.
 11. A non-transitory computer readable storagemedium storing a plurality of instructions in connection with a terminaldevice, wherein the plurality of instructions, when executed by one ormore processors of the terminal device, cause the terminal device toperform a live video broadcast method including: receiving a livebroadcast command to live broadcast a video game in real time; creatinga video buffer based on the live broadcast command for storing one ormore game pictures of the video game in real time; while playing thevideo game in real time: detecting and extracting a plurality of videopicture frames from the video game; storing the plurality of videopicture frames in the video buffer; displaying the plurality of videopicture frames in the video buffer on a display of the terminal device;collecting voice data using a microphone of the terminal device in realtime; synchronously synthesizing the voice data and the plurality ofvideo picture frames stored in the video buffer into a video streamingmedia file corresponding to the video game; and uploading the videostreaming media file to a live broadcast server for live broadcastingthe video streaming media file on other mobile devices.
 12. Thenon-transitory computer readable storage medium according to claim 11,wherein the operation of creating a video buffer based on the livebroadcast command comprises: invoking a pre-configured mountingcomponent based on the live broadcast command; replacing a pictureloading method using the mounting component; and creating acorresponding video buffer based on the replaced picture loading method.13. The non-transitory computer readable storage medium according toclaim 12, wherein the operation of replacing a picture loading methodusing the mounting component comprise: detecting whether the pictureloading method is invoked; and replacing a picture drawing method in thepicture loading method when the picture loading method is invoked, so asto detect updating of the video picture frame by using the game engine.14. The non-transitory computer readable storage medium according toclaim 11, wherein the live broadcast server is configured to perform thelive broadcasting by: receiving comment information associated with thevideo game from the other mobile devices; synthesizing the commentinformation and the video streaming media file, to obtain a livestreaming media file with comments; and performing a broadcast operationon the live streaming media file with comments to the other mobiledevices, wherein a respective mobile device is configured to display thecomments from other mobile devices that are currently live broadcastingthe video game.
 15. The non-transitory computer readable storage mediumaccording to claim 11, wherein the live broadcast server is configuredto perform the live broadcasting by: receiving barrage commentsinformation associated with the video game from the mobile devices;performing a broadcast operation on the video streaming media file; andsynchronously superimposing the barrage comments information on theassociated video picture frames of the video streaming media file on aplayback screen of the terminal device.